Lost Relatives of the Gumbel Trick

APPENDIX: Lost Relatives of the Gumbel Trick Here we provide proofs for the results stated in the main text, together with additional supporting lemmas required for these proofs. A. Comparison of Gumbel and Exponential tricks In Section 2.3.1 we analyzed the asymptotic efficiency of different estimators of Z by measuring their asymptotic variance. (As all our estimators in the full-rank perturbation setting are consistent, their bias is 0 in the limit of infinite data, and so this asymptotic variance equals the asymptotic MSE.) In the non-asymptotic regime, where an estimator ˆ Z is constructed from a finite set of M samples, we can analyze both the variance var( ˆ Z) and the bias (E[ ˆ Z] Z) of the estimator. While in most cases these cannot be obtained analytically and there we can resort to an empirical evaluation, for the estimators stemming from the Gumbel and Exponential tricks analytical treatment turns out to be possible using standard methods. A.1. Estimating Z Gumbel trick The Gumbel trick yields an unbiased estimator for lnZ, and we can turn it into a consistent estimator of Z by exponentiating it: